Error
Unrecognized LW server error:
Field "fmCrosspost" of type "CrosspostOutput" must have a selection of subfields. Did you mean "fmCrosspost { ... }"?Unrecognized LW server error:
Field "fmCrosspost" of type "CrosspostOutput" must have a selection of subfields. Did you mean "fmCrosspost { ... }"?
I fixed a bug that was causing this post to get underweighted by the frontpage algorithm, and have reset the date of posting to now, to correct for the period where it wouldn’t have showed up on the frontpage.
Thanks!
Hi! You might want to start putting these in a sequence, as Vaidehi and I have done: 2017, 2018, 2019
Thanks for the suggestion(s)! I was waiting for us to have more of the posts in the series out. But, now that there are only a couple left, I agree it’s time.
Is it possible to get the ‘importance for involvement’ for all the four options of Gender X Engagement? I’d like to understand whether the high engagement of people marked ‘personal contact’ or ‘group’ as important for their involvement could be partially explained by their gender, or something of this sort. Doing that with Race could also be interesting.
I think this is what you want?.
(R note: I haven’t figured out how to purrr:map list splits of the dataframe to ggplot so I can’t automate these easily yet)
Thanks, yea!
Going over this, I don’t see anything particularly interesting. It looks like the ratio of people being highly engaged to not highly engaged per each factor is about the same for males and females in almost all categories. Some of the slight differences that I could find:
Males who rated EAG as important were about twice as likely to be not highly engaged compared to non-males (though the error is high here).
The share of not highly engaged non-males which had ‘personal connection’ as an important factor for involvement was slightly higher than the male counterpart. This slightly reduces the gap between males and non-males when it comes to how important is ‘personal connection’ for involvement for people who are highly engaged.
[Epistemic status: just looked into this briefly out of curiosity, not an official EAS analysis]
When I looked at this briefly in a generalized mixed model, I didn’t find a significant interaction effect for gender * engagement * the specific factor people were evaluating (e.g. EAG or group etc.) which comports with your observation that there doesn’t seem to be anything particularly interesting going on in the contrast between male and non-male interaction with low/high engagement. (In contrast, there were significant fixed effects for the interaction between engagement and the specific factor and gender and the specific factor.) Looking at the specific ‘getting involved factors’ in the interaction effect, it was clear where the only one where there was much of a hint of any interaction with gender * engagement was personal contact, which was “borderline significant” (though I am loathe to read much into that).
Probably the simplest way to illustrate the specific thing you mentioned is with the following two plots: looking at both male and non-male respondents, we can see that highly engaged respondents are more likely to select EAG than less engaged respondents, but the pattern is similar for both male and non-male respondents.
IMO it is hard know what inference to draw from these comparisons.
Firstly, making multiple comparisons obviously raises the risk of a “false-positive” … a result that is merely due to chance/sampling.
Secondly, with ‘multiple hurdles’ it’s hard to know how to compare like for like....
--> But note that the involvement factors may be driving engagement itself, and doing so differently for males and females
Also note that females tend to be less engaged, although the differences are not extremely large. See GRAPHS HERE—that link is a preview of results we will put out in our forthcoming ‘Engagement’ post.
Thanks, looking forward for the full post :)
It looks like Sam Harris interviewed Will MacAskill this year. He also interviewed Will in 2016. How might we tell if the previous interview created a similar number of new EA-survey-takers, or if this year’s was particularly successful? The data from that year https://forum.effectivealtruism.org/posts/Cyuq6Yyp5bcpPfRuN/ea-survey-2017-series-how-do-people-get-into-ea doesn’t seem to include a “podcast” option.
We show changes in the proportion of respondents coming from each source across cohorts using this year’s data here.
You can see the increase in absolute numbers coming from Podcasts and the % of each cohorts coming from Podcasts below. Because some portion of each cohort drop out every year, this should give an inflated impression of the raw total coming from the most recent cohort (2020) compared to earlier cohorts though. Comparing raw totals across years is not straightforward, because sample size varies each year (and we sampled fewer people in 2020 than earlier years as discussed here and here and although we think we can estimate our sampling rate for engaged EAs quite well, we’re less certain about the true size of the more diffuse less engaged EA population (see here))- so the totals for ~2017 at the time were likely relatively higher.
Sorry, I neglected to say thank you for this previously!
Thanks!
Incidentally your comment just now prompted me to look at the cross-year cross-cohort data for this. Here we can see that in EAS 2019, there was a peak in podcast recruitment closer to 2016 (based on when people in EAS 2019 reported getting involved in EA). Comparing EAS 2019 to EAS 2020 data, we can see signs of dropoff among podcast recruits among those who joined ~2014-2017 (and we can also see the big spike in 2020).
These are most instructive when compared to the figures for other recruiters (since the percentage of a cohort recruited by a given source is inherently a share relative to other recruiters, i.e. if one percentage drops between EAS 2019 and EAS 2020 another’s has to go up).
Comparing personal contact recruits we can see steadier figures across EAS 2019 and EAS 2010, suggesting less dropoff. (Note that the figures for the earliest cohorts are very noisy since there are small numbers of respondents from those cohorts in these surveys).
Thanks for these very helpful insights! I thought the mosaic charts were particularly creative and visually insightful.
I have one minor statistical nit and one related question.
In cases where ‘only one significant difference was found’ (at a 95% c.i.), it could be worth noting that you have around 20 categories… so on average one spurious significant difference is to be expected! (If the difference is small.)
Also a question about how the significance test was carried out.σ(a−b)=√σ(a)2+σ(b)2≤σ(a)+σ(b) so for calling a difference significant at 95% it matters whether you a) check if the individual 95% confidence intervals overlap or b) check if the diff’d confidence interval noted above contains 0 (the usual approach). Which approach was used here? I ask because to my eye there might be a few more (weakly) significant results than were mentioned in the text.
Hi Oliver. Thanks for your question!
We actually just performed the same analyses as we did last year, so any references to significance are after applying the Bonferroni adjustment. We just decided to show the confidence intervals rather than just the binary significant/not significant markers this year, but of course different people have different views about which is better.